Federated learning (FL) has emerged as an instance of distributed machine learning paradigm that avoids the transmission of data generated on the users' side. Although data are not transmitted, edge devices have to deal with limited communication bandwidths, data heterogeneity, and straggler effects due to the limited computational resources of users' devices. A prominent approach to overcome such difficulties is FedADMM, which is based on the classical two-operator consensus alternating direction method of multipliers (ADMM). The common assumption of FL algorithms, including FedADMM, is that they learn a global model using data only on the users' side and not on the edge server. However, in edge learning, the server is expected to be near the base station and have direct access to rich datasets. In this paper, we argue that leveraging the rich data on the edge server is much more beneficial than utilizing only user datasets. Specifically, we show that the mere application of FL with an additional virtual user node representing the data on the edge server is inefficient. We propose FedTOP-ADMM, which generalizes FedADMM and is based on a three-operator ADMM-type technique that exploits a smooth cost function on the edge server to learn a global model parallel to the edge devices. Our numerical experiments indicate that FedTOP-ADMM has substantial gain up to 33\% in communication efficiency to reach a desired test accuracy with respect to FedADMM, including a virtual user on the edge server.
translated by 谷歌翻译
光子计数CT(PCCT)通过更好的空间和能量分辨率提供了改进的诊断性能,但是开发可以处理这些大数据集的高质量图像重建方法是具有挑战性的。基于模型的解决方案结合了物理采集的模型,以重建更准确的图像,但取决于准确的前向操作员,并在寻找良好的正则化方面遇到困难。另一种方法是深度学习的重建,这在CT中表现出了巨大的希望。但是,完全数据驱动的解决方案通常需要大量的培训数据,并且缺乏解释性。为了结合两种方法的好处,同时最大程度地降低了各自的缺点,希望开发重建算法,以结合基于模型和数据驱动的方法。在这项工作中,我们基于展开/展开的迭代网络提出了一种新颖的深度学习解决方案,用于PCCT中的材料分解。我们评估了两种情况:一种学识渊博的后处理,隐含地利用了模型知识,以及一种学到的梯度,该梯度在体系结构中具有明确的基于模型的组件。借助我们提出的技术,我们解决了一个具有挑战性的PCCT模拟情况:低剂量,碘对比度和很小的训练样品支持的腹部成像中的三材料分解。在这种情况下,我们的方法的表现优于最大似然估计,一种变异方法以及一个完整的网络。
translated by 谷歌翻译
我们提出了层饱和 - 一种简单的在线可计算的方法,用于分析神经网络中的信息处理。首先,我们表明层的输出可以限制在没有性能损失的方差矩阵的eIgenspace。我们提出了一种计算上的轻量级方法,用于在训练期间近似方差矩阵。从其无损EIGenspace的维度我们推导了层饱和度 - eIGenspace尺寸和层宽度之间的比率。我们表明饱和度似乎表明哪个层有助于网络性能。我们通过改变网络深度,滤波器大小和输入分辨率,展示如何改变神经网络中的层饱和度。此外,我们表明,通过在网络上更均匀地分配推动过程,所选择的输入分辨率提高了网络性能。
translated by 谷歌翻译